In large distributed storage systems, erasure coding has been used to minimize cost and increase reliability. Erasure coding generally is a technique by which data is broken into chunks, encoded with redundant pieces of data and stored across different nodes. If a particular chunk of data is lost, then it can be recovered by decoding the remaining chunks of data. Data intensive applications, such as big data analytics, multimedia streaming, ecommerce, and social media have benefited from the use of erasure coding.
Nevertheless, the popularity of such applications and the corresponding increase in demand has placed a burden on existing data storage systems. Simply put, the sheer amount of traffic is making it more and more difficult to transfer data from storage to end users in a timely manner. One conventional way of reducing this type of latency is through the use of caching. By storing large chunks of popular data in high speed storage and/or at locations near end-users, caching reduces congestion and decreases the time for processing file requests. However, to date, there are no efficient mechanisms for caching data that has been erasure coded. Accordingly, what is needed is a functional caching approach for erasure coded storage as set forth herein.